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ABSTRACT 

This  report  summarizes  the  results  of  five  years  of  research  in  the 
general  areas  of  algebraic  systems  and  stochastic  systems.  Topics  con¬ 
sidered  related  to  algebraic  systems  include:  unequal  error  protection 
codes,  burst  and  multiple  burst  correcting  codes,  codes  designed  for  the 
Lee  metric,  improved  BCH  codes,  decoding  of  BCH  codes,  system  reliability 
through  algebraic  coding,  channel  modeling  of  real  channels,  the  use  of  a 
time  dispersive  channel  as  an  encoder  and  the  study  of  certain  algebraic 
sequences.  In  the  study  of  stochastic  systems  the  following  topics  were 
considered:  optimum  sampling,  invariant  estimators,  feedback  communica¬ 
tions,  adaptive  detection  and  learning,  high  speed  binary  data  transmis¬ 
sion  over  a-band-llmited  channel,  multiple  access  communications  and 
rate  distortion  functions  for  stochastic  sources. 

Only  a  brief  summary  of  the  research  results  is  presented.  However, 
abstracts  of  papers  and  dissertations  resulting  from  this  research  are 


included . 


I. 


introduction 


This  is  the  final  report  on  Contract  Ar49(63B)-1600  undertaken  with 
the  Air  force  Office  of  Scientific  Research.  United  States  Air  force. 

This  contract  was  in  effect  from  September  16,  1965  to  September  15,  1970. 
Various  research  topics  were  studied  in  the  general  area  of  communications 
theory  and  information  theory.  This  research  was  also  concerned  with  the 
relationship  of  these  areas  to  automata  and  control  theory. 

In  this  section,  a  brief  summary  of  the  research  completed  is  pre¬ 
sented;  abstracts  of  published  papers  and  dissertations  which  are  concerned 
with  these  research  topics  are  given  in  later  sections.  Detailed  discus¬ 
sions  of  this  research  are  contained  in  the  actual  published  papers  and 
dissertations. 

The  research  areas  investigated  fall  into  two  categories  which  will 
be  termed  algebraic  systems  and  stochastic  systems.  The  studies  in  alge¬ 
braic  systems  include  several  topics  in  alpehraic  coding  as  well  as 
studies  of  algebraic  sequences  with  desirable  properties.  The  algebraic 
codes  studied  have  applications  in  the  reliable  transmission  of  digital 
data  over  noisy  channels,  the  reliable  storage  and  recovery  of  data  from 
high  density  storage  media  and  uvta  compression.  The  algebraic  sequences 
considered  have  applications  in  wide  band  communications,  multiple  access 
communications ,  multiplexing,  addressing,  pattern  recognition  and  en¬ 
cipherment  . 

The  studies  in  stochastic  systems  include  several  topics  in  signal 
detection  and  estimation,  signal  design,  feedback  communications,  chan¬ 
nel  modeling  and  learning  machines. 
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Algebraic  Systems 

Various  methods  of  algebraic  coding  were  considered  in  this  investi¬ 
gation.  These  include  unequal  error  protection  codes,  burst  and  multiple 
burst  correcting  codes,  codes  based  upon  interleaved  algebraic  sequences, 
codes  designed  for  the  Lee  metric,  improved  PCH  codes,  methods  of  decoding 
F-CH  codes  and  methods  of  improving  the  r  liability  of  digital  systems  by 
coding. 

Unequal  error  protection  codes  have  the  capability  of  providing  dif¬ 
ferent  levels  of  error  protection  for  specific  digits  of  a  code  word. 

Such  codes  may  haw  application  in  PCM  transmission  of  quantized  sampled 
data  where  errors  in  certain  digits  would  be  more  costlv  in  reconstruction 
than  errors  in  other  digits.  Methods  of  synthesizing  such  codes  and  de¬ 
coding  algorithms  have  been  found.  Upper  and  lower  bounds  on  the  redun¬ 
dancy  have  also  been  derived.  (See  II-2  and  TV-8) 

Several  classes  of  single  burst  and  multiple  burst  correcting  algebraic 
codes  have  been  studied.  The  concept  of  a  burst  distance  was  defined  and 
the  burst  correcting  capabilities  of  codes  was  related  to  the  burst  dis¬ 
tance  of  the  codes.  The  multiple  burst  correcting  capability  of  product 
codes  was  investigated.  This  class  of  codes  is  particularly  interesting 
because  of  an  easy  decoding  algorithm  developed  for  them.  (Fee  TTT-f  and 
IV-17.) 

A  new  class  of  nonlinear  random  error  correct inp  codes  based  up  ' 
the  interleaving  of  periodic  algebraic  seouences  w*s  developed.  The 
performance  of  such  codes  is  roughlv  the  same  as  the  best,  presently 
known,  codes,  the  BUH  codes.  (See  IV-4./ 
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A  study  was  made  of  codes  desired  for  the  Lee  (rather  than  the 
Hamming)  metric.  Such  codes  would  appear  to  have  application  in  a 
M-phase  or  M-amplitude  communications  system  where  M  is  a  prime  greater 
tha.i  2.  General  properties  of  such  codes  have  been  derived.  Also  spe¬ 
cific  codes  have  been  synthesized  for  correcting  random  errors  or  burst 
errors.  (See  III-9.) 

An  interesting  duality  was  discovered  between  a  decoding  a.  ^orithm 
for  PCH  codes  and  a  common  method  of  curve  fitting.  (See  IV-7.)  Also 
it  was  found  that  two  information  symbols  can  be  appended  to  certain 
PCH  codes  without  effecting  the  guaranteed  minimum  distance  of  these 
codes.  The  lengthened  codes  are  then  used  as  building  blocks  for  per¬ 
fect  single-error  correcting  codes  in  another  metric.  (See  IV-14.) 

The  techniques  of  algebraic  coding  theorv  were  used  to  improve  the 
reliability  of  digital  devices.  Redundant  circuits  were  designed  and 
evaluated  based  upon  different  classes  '  '  algebraic  codes.  (See  IV-15.) 

Finally,  two  investigations  were  concerned  with  the  use  of  codes 
over  real  communication  channels.  "Hie  first  of  these  was  concerned 
with  developing  a  model  for  the  errors  which  occur  on  a  real  channel. 

A  model  was  developed  which  could  match  the  error  distributions  recorded 
from  different  tvnes  of  channels,  for  which  the  parameters  of  the  model 
could  be  easily  determined  and  for  which  the  performance  of  codes  could 
he  evaluated  quite  simply.  (See  ITI-3  and  IV-10.)  The  second  investi¬ 
gation  was  concerned  with  the  similarities  between  a  time  dispersive 
channel  and  an  encoder.  In  this  investigation,  a  time  dispersive  chan¬ 
nel  was  equalized  so  as  to  set  as  an  encoder  for  an  algebraic  code  -- 
u.ut  is,  to  produce  the  check  symbols  from  the  input  message  digits. 
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The  received  digits  are  then  decoded  bv  an  ordinary  algebraic  decoder. 
Computer  simulations  indicate  that  under  certain  conditions,  orders  of 
magnitude  improvement  on  error  rate  can  be  achieved  by  this  method  over 
the  usual  methods  of  transmission.  (See  III-10.) 

Three  studies  were  conducted  of  algebraic  sequences  with  certain 
desirable  properties.  First,  several  sets  of  sequences  were  synthe¬ 
sized  for  which  everv  pair  of  the  set  has  a  zero  cross-correlation  for 
all  time  shifts.  These  seauences  were  applied  to  the  problem  of  design- 
inp  carriers  for  an  asynchronous  multiplex  system.  (See  IV-2.  and  1V-3.) 

An  investigation  of  the  properties  of  subsequences  of  lonp  pseudo¬ 
random  sequences  was  completed.  These  properties  in  the  form  of  moments 
of  the  weight  distribution  are  shown  to  provide  a  method  for  selecting 
pood  pseudo-random  sequences  for  correlation-detection  applications. 

( See  TT-6.  ) 

Also,  the  synthesis  of  two-dimeiv  ioi  -.1  >•  <-s  exhibiting  good 

two-dimensional  autocorrelation  properties  and  cross -correlation  was 
accomplished.  Special  arravs  were  constructed  that  had  two- level  auto¬ 
correlation  functions.  These  arravs  mav  have  applications  in  optical 
signal  processing  and  pattern  recognition.  (See  IT-1  and  IV-9.) 
Stochastic  Systems 

Two  problems  were  considered  In  the  area  of  identification  of  un¬ 
known  systems  or  processes.  An  optimum  sampling  rate  was  found  for 
modeling  a  continuous  time  system  bv  a  discrete  time  system  model.  It 
was  shown  that  the  common  assumptions  that  faster  samrlinr  leads  to 
better  results  Is  not  true  in  general  and  that  in  at  least  one  sense  an 
optimum  sampling  rate  exists  for  a  given  tvre  of  model.  (Fee  Tw-1). 
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A  second  problem  was  concerned  with  finding  an  estimator  of  the  autocorre¬ 
lation  function  such  that  the  average  risk  associated  with  the  estimate 
is  invariant  to  the  true  values  of  the  autocorrelation  (See  III-l  and 

TV-5 . ) 

Two  investigations  were  conducted  in  the  area  of  feedback  communica¬ 
tions.  A  sequential  coding  scheme  was  analvzed  and  compared  with  the 
conventional  block  coded  scheme  for  both  wideband  and  bandlimited  channels. 
(See  III -2 . )  Also  a  feedback  scheme  based  upon  the  Kiefer-Wolfowitz 
stochastic  approximation  technique  was  analvzed.  It  was  shown  that  for 
the  wideband  case  this  scheme  performed  the  same  as  the  usual  scheme 
based  upon  the  Robbins-Monro  stochastic  approximation  procedure.  (See 
TI-4.  ) 

Adaptive  detection  and  learning  machines  were  the  subiect  of  two 
related  studies.  A  comparison  of  a  taught,  untaught  and  decision  di¬ 
rected  detectors  was  made.  (See  II-3.)  Also  an  adaptive  detection 
scheme  based  upon  extreme  value  statistics  was  postulated  and  analvzed. 

This  scheme  used  the  t^pry  of  extremes  to  determine  the  behavior  of 
the  underlying  probability  density  functions  on  their  tails.  From  this 
information  the  best  threshold  was  determined.  In  the  case  of  a  fading 
signal,  the  receiver  became  adaptive  with  a  time  va. ,ing  threshold. 

(See  III-4  and  IV-13.) 

The  transmission  of  the  linear  sum  of  m>2WT  biphase  modulated 
signals  in  a  time  interval  T  was  considered  for  an  additive  naussian 
white  noise  channel  of  bandwidth  V.  it  was  found  that  if  all  m  signals 
were  independently  modulated  the  error  probability  can  be  made  to  go  to 
zero  exponentially  with  2VT  if  the  signal  to  noise  ratio  increases  lin 
early  with  2VT.  It  was  also  abwn  that  if  k  m  are  independently  modu- 
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lated  the  remainder  carving  redundant  information,  for  a  fixed  signal 
to  noise  ratio  the  error  probability  can  be  made  to  go  to  zero  for 

ft  ft 

k/2WT<C  .  For  high  signal  noise  ratios  C  is  greater  than  1,  (See 
III -5 ,  IV-6  and  TV-12.) 

The  asymptotic  probability  of  error  for  the  transmission  of  a  large 
number  of  orthogonal  signals  over  a  generalized  incoherent  channel  was 
also  investigated.  (See  II I -5  and  IV-11).  Another  investigation  was 
concerned  with  bounds  for  the  probabilitv  of  error  for  a  binary  input 
Gaussian  channel  where  the  output  is  quantized  to  Qlevels.  Three  sit¬ 
uations  were  considered:  no  feedback,  decision  feedback  and  information 
feedback.  (See  II I -7.) 

Stochastic  sources  were  also  investigated.  The  rate-distortion 
function  for  a  Caussian-Markov  source  was  investigated  for  a  mean-square 
distortion  measure.  It  is  shown  that  if  a  feedback  channel  Is  available, 
the  Ideal  performance  can  be  achieved  using  a  simple  scheme.  The  ideal 
performance  is  onlv  achieved  with  infinite  delav.  The  rate  of  conver¬ 
gence  to  the  minimum  mean-square  error  as  a  function  of  delav  was  found 
for  both  feedback  and  nonfeedback  schemes.  A  differential  PfV  and  an 
adaptive  quantizing  system  were  found  to  perform  within  4.34dh  of  the 
rate  distortion  function  bound.  (See  ITT-B,  TV-lf  and  TV-IP.) 

Finally,  the  subject  of  probabilistic  decoding  was  considered.  In 
particular,  the  performance  of  a  multiple-access  communication  system 
was  computed  with  probabilistic  decoding  used  on  the  links.  Performance 
curves  showing  tradeoffs  between  power  and  bandwidth  were  calculated. 


(See  II-5 . ) 
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The  objectives  of  university  research  are  varied.  Two  of  the  prime 
objectives  are  the  education  of  new  research  workers  and  the  pursuit 
of  new  knowledge.  It  is  felt  that  both  of  these  objectives  were  achieved 
under  this  contract. 

During  the  past  five  years,  six  master's  projects  and  ten  doctoral 
dissertations  were  completed  under  sponsorship  (or  partial  sponsorship) 
of  this  contract.  Other  doctoral  dissertations  are  currently  in  progress 
but  were  not  reported  here.  These  students,  the  project  director  and 
the  Polytechnic  Institute  of  Brooklyn  are  deeply  appreciative  to  the  Office 
of  Scientific  Research  of  the  United  States  Air  force  for  the  essential 
part  they  have  played  in  this  propram. 
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II.  MASTER'S  REPORTS  f 

II-l.  "Correlation  of  Doubly  Periodic  Arrays" 
by  Domenick  Calabro  -  1967 

Abstract 


This  report  is  concerned  with  the  correlation  properties 
of  two  dimensional  arrays  with  binarv  and  nonbinary  elements. 
The  arrays  are  formed  bv  repeating  a  basic  block  of  elements 
on  all  four  sides  and  diagonally  producing  a  doublv  periodic 
structure.  The>  correlation  function  of  these  arrays  now  in¬ 
volves  two  dimensional  shifting  and  addition  operations.  The 
ultimate  goal  is  to  synthesize  arrays  exhibiting  prescribed 
correlation  properties. 

The  first  part  of  this  report  concerns  the  synthesis  of 
binary  and  nonbinarv  (N-arv)  arrays  exhibiting  two  level  auto¬ 
correlation  functions.  In  the  second  part  of  this  report 
arrays  exhibiting  perfect  periodic  cross-correlation  functions 
(zero  cross-correlation  for  all  cyclic  shifts)  and  good  auto¬ 
correlation  functions  are  constructed. 
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II-2.  "Linear  Coding  Techniques  for  Unequal  Error  Protection" 
by  Gerald  Pasternack  -  1967 

Abstract 

Although  intensive  research  is  being  conducted  in  the  area 
of  linear  coding  theory,  comparatively  little  concerns  itself 
with  "Unequal  Error  Protecting"  (UEP)  codes.  These  are  codes 
which  have  the  capability  of  providing  different  levels  of  error 
protection  for  specific  digits  of  the  code  word  -  as  opposed  to 
the  more  conventional  "Equal  Error  Protecting"  (EEP)  codes  in 
which  the  digits  of  the  coded  word  are  all  protected  to  the  same 
degree . 

This  paper  presents  several  results  pertinent  to  UEP  codes. 
A  theoretical  basis  for  these  codes  ana  their  relation  to  EEP 
codes  has  Lien  developed  by  other  authors  and  these  results  are 
briefly  discussed.  Next,  an  algorithmic  technique  for  code 
construction  is  established.  By  usinp  this  procedure,  a  spe¬ 
cific  UEP  code  of  minimum  length  may  be  constructed.  The  meth¬ 
ods  of  integer  linear  programming  are  employed  and  the  applica¬ 
tion  of  the  "simplex  algorithm"  is  discussed  in  detail  and 
illustrated.  Upper  bounds  on  both  the  computational  complexity 
of  the  simplex  algorithm,  as  well  a3  code  parameters  are  de- 
rived„  Finally,  the  problem  of  decoding  is  investigated,  and 
the  theoretical  basis  for  two  iterative  techniques  is  developed. 

Each  of  the  concepts  presented  are  illustrated  with  prac¬ 
tical  example  • , 


? 
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IT-3.  "Some  Learning  Machines  for  Pattern  Recognition" 
by  John  Robert  Clark  -  1968 

Abstract 

In  this  report  we  examine  some  basic  pattern  recognition 
machines  with  the  learning  feature.  The  discussion  is  res¬ 
tricted  to  parametric  learning  of  the  unknown,  constant  mean 
vector  of  a  pattern  class. 

A  machine  is  formulated  in  general  terms,  and  methods 
for  realizing  various  learning  algorithms  are  indicated. 

Both  Bayesian  decision  theory  and  stochastic  approximation 
are  used  in  the  derivations.  A  specific  example  -  a 
single-sample  binary  detector  for  an  unknown  signal  in  ad¬ 
ditive,  white,  gaussian  noise  -  is  examined  in  detail. 
Performance  figures  are  obtained  whenever  possible  for  var¬ 
ious  learning  algorithms:  taught,  untaught,  decision- 
directed,  and  combination  taught-decision-directed.  Faeh 
results  are  obtained  in  the  performance  analyses  of  the 
taught  and  untaught  machines,  but  only  approximate  results 
are  obtained  for  the  other  machines.  However,  a  straight¬ 
forward  but  tedious  procedure  for  finding  exact  answers 
is  outlined.  The  taught  machine  is  found  to  be  the  best 
in  almost  all  respects,  followed  closelv  by  the  decision- 
directed  machine  at  all  but  the  poorest  signal-to-noise 
ratios.  The  untaught  machine  is  more  appealing  than  the 
decision-directed  detector  at  low  signal-to-noise  ratios, 
where  the  latter  becomes  strongly  biased  in  the  estimate 
of  the  decision  threshold. 

Results  of  computer  simulations  are  presented;  these 
show  good  agreement  with  the  theoretical  predictions. 
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1-4.  "A  Feedback  Communication  Scheme  Based  on  the 
Kiefer-Wolfowitz  Procedure" 
by  Richard  W.  Muise  -  1968 

Abstract 


A  feedback  communication  scheme  has  been  studied  by  Kailath 
and  Schalkwijk  which  is  based  on  the  Robbins-Monro  stochastic 
approximation  technique.  They  assumed  that  the  channel  consisted 
of  a  noiseless  feedback  link  and  an  additive  white  Gaussian  noise 
forward  link.  The  analysis  was  treated  in  two  parts,  first  with 
no  bandwidth  constraint  on  the  signals  and  second  with  a  finite 
bandwidth  constraint  on  the  signals.  This  report  discusses  a 
feedback  cormunicatior  scheme  subject  to  the  same  assumptions 
and  constraints  but  based  on  the  Kiefer-Wolfowita  stochastic 
approximation  technique.  It  is  shown  that  the  performance  ach¬ 
ieved  with  this  scheme  is  the  same  as  that  achieved  with  the 
Schalkwi jk-Kailath  scheme  for  no  bandwidth  constraint.  However, 
then  a  finite  bandwidth  constraint  is  imposed  on  the  signals,  the 
Schalkwi jk-Kailath  scheme  is  shown  to  perform  better,  with  the 
difference  in  performance  becoming  arbitrarily  small  as  the 
bandwidth  is  allowed  to  increase  without  limit. 

The  two  feedback  schemes  are  then  compared  with  the  best 
known  (simplex  signals)  one  way  scheme  and  are  shown  fo  have 
better  performance  than  the  one  wav  scheme. 
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II-5.  "Coding  for  Multiple-Access  Communication  Systems" 
by  Nicola  A.  Macina  -  1969 

Abstract 


To  provide  a  communication  system  with  multiple-access 
capability  imposes  certain  sacrifices  in  overall  system 
efficiency;  for  a  given  amount  of  information  to  be  exchanged 
over  the  system,  more  power  and  bandwidth  are  required  than 
for  more  conventional  systems.  Of  the  many  techniques  pro¬ 
posed  and  analyzed  over  the  past  years,  a  few  seem  to  benefit 
from  the  application  of  coding  theory  to  their  design.  This 
report  analyzes  one  of  these  techniques ,  a  Pulse-Address 
Multiple  Access  (PAMA)  system,  for  the  purpose  of  showing  the 
extent  to  which  coding  can  improve  the  system  efficiency  in 
power  and  bandwidth  utilization.  A  system  model  is  first 
defined,  patterned  after  a  satellite  communication  system, 
and  the  performance  of  a  non-coherent  block-orthogonal  signal¬ 
ing  scheme  is  calculated.  Next,  algebraic  coding  is  applied 
and  shown  to  imp/  ve  the  performance  significantly,  particu¬ 
larly  in  bandwidth,  where  a  reduction  by  almost  three  orders 
of  magnitude  is  achieved.  Finally,  probabilistic  coding 
(sequential  decoding)  is  applied,  and  two  deciding  strate¬ 
gies  analyzed:  Hard  decision,  and  detection  with  list.  Fur¬ 
ther  improvements  over  algebraic  coding  are  realized  by 
amounts  generally  commensurate  to  the  mechanization  complexity. 
For  each  case,  performance  curves  are  presented  as  tradeoffs 
between  power  and  bandwidth  requirements  to  guide  the  designer 
in  the  choice  of  the  best  engineering  compromises.  A  practical 
example  of  the  application  of  the  results  of  the  analysis  con¬ 
cludes  this  report.  This  study  is  considered  significant,  as 
it  presents  a  systematic  approach  to  a  communication  systf 
design,  while  proving  the  effectiveness  of  coding  to  improve 
the  efficiency  of  a  particular  category  of  communication 
systems . 
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II-6.  "Properties  of  Subsequences  of  Pseudo-Random  Sequences" 
by  S.  Wainberg  -  1969 

Abstract 


One  method  of  synchronizing  a  communication  system  uses 
the  cross correlation  between  a  maximal  length  pseudorandom, 
binary  sequence  and  a  portion  of  that  sequence.  Using  only 
part  of  the  sequence  causes  the  correlation  function  to  have 
peaks  when  the  sequences  are  not  aligned,  and  thus  may  cause 
A  threshold  decision  scheme  to  synchronise  incorrectly.  The 
object  of  this  report  is  to  investigate  the  properties  of  the 
svbsequences  of  long  psuedorandom ,  binary  sequences.  These 
properties,  in  the  form  of  the  moments  of  the  weight  distri¬ 
bution  of  the  subsequences,  are  shown  to  provide  a  practical 
aid  for  selecting  good  m-sequences  for  various  correlation- 
detection  design  problems.  In  particular,  the  first  four 
moments  are  described  in  detail  for  subsequence  lengths  up 
to  one  hundred  for  six  different  m-sequences.  Using  the 
moments,  a  relationship  is  shown  between  the  subsequences 
of  the  pseudorandom  binary  sequence  and  subsequences  composed 
of  random,  binary  numbers.  A  technique  is  shown  for  finding 
the  characteristic  polynomials  or  generating  functions  which 
will  produce  an  m-sequence  that  has  subsequence  properties 
similar  to  that  of  random  number  sequences.  An  attempt  is 
made  to  determine  the  value  of  sampling  the  m-sequence  to 
find  the  moments  of  the  weight  distribution  of  the  subsequen¬ 
ces.  Again  a  comparison  is  made  to  the  random  number  situa¬ 
tion.  These  investigations  enable  us  to  model  some  m- 
sequences  by  random,  binary,  numbers. 
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III.  Ph.D.  DISSERTATIONS 

III-l.  "Invariant  Estimation  of  Stochastic  Systems  Parameters" 
by  Guner  Suzek  -  1966 

Abstract 

This  dissertation  is  concerned  with  the  estimation  of  the 
statistics  of  a  class  of  random  processes.  Properties  of  a 
new  constant  risk  estimator  of  the  autocorrelation  function 
for  exponentially  correlated  processes  are  studied  analytically. 
The  results  are  applied  to  computer-generated  random  processes 
and  the  estimates  found  are  compared  with  the  results  of  more 
common  estimation  methods,  such  as  autoregressive  estimation. 

The  effect  of  increasing  the  amount  of  data  and  changing  the 
input  distribution  is  examined  both  analytically  and  experi¬ 
mentally. 
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II I -2 .  "Some  Results  for  Additive  Noise  Channels  with  Noiseless 
Information  Feedback" 
by  Thomas  W.  Eddy  -  1968 

Abstract 


This  thesis  presents  some  new  results  for  two  distinct 
aspects  of  the  additive  noise  channel  with  noiseless  informa¬ 
tion  feedback.  First,  the  performance  capabilities  of  chan¬ 
nels  with  additive  colored  noise  in  the  forward  channel  are 
examined.  Next,  a  sequential  coding  scheme  is  proposed  for 
the  additive  white  noise  channel  and  its  performance  is  anal¬ 
yzed  and  compared  to  the  performance  of  the  Schalkwijk  coding 
procedures.  The  coding  procedures  analyzed  in  both  segments 
of  the  thesis  employ  the  basic  ideas  of  the  Schalkwiik  coding 
schemes . 


In  the  last  part  of  the  thesis  a  sequential  coding  pro¬ 
cedure  using  information  feedback  is  proposed  and  analyzed. 
Performance  curves  for  wideband  and  bandlimited  channels  are 
presented  and  the  asymptotic  probability  of  error  is  deter¬ 
mined.  For  tne  wideband  channel 
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where  T  is  the  average  time  per  message,  R  =  In  M/T,  C  is 
the  channel  capacity  and  v  =  0.5772156649...  is  Euler's 
constant.  Similarly,  for  the  bandlimited  channel 
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While  the  error  performance  is  significantly  improved  by 
using  sequential  information  feedback,  the  sequential  results 
are  valid  only  if  an  infinite  buffer  is  used.  In  the  last 
chapter  the  effect  of  a  finite  buffer  is  examined.  It  is 
show,  (for  a  particular  algorithm)  that  the  finite  buffer  in¬ 
troduces  an  additional  probability  of  error  which  decreases 
as  1/  where  B  is  the  maximum  number  of  messages  that  can 
be  stored  in  the  buffer.  This  result  illustrates  that  the 
significant  role  played  by  the  buffer  in  the  sequential  coding 


procedure . 
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III-3.  "Error  Control  on  Real  Channels" 
by  Michael  Muntner  -  1968 

Abstract 


The  promise  of  significant  improvement  associated  wi;h 
the  use  of  error  control  techniques  on  digital  communications 
channels  has  been  based  upon  the  assumption  that  errors  occur 
independently.  Real  channels  (e.g.,  telephone  circuits, 
troposcatter  radio,  etc. )  however  do  not  exhibit  this  property 
in  that  errors  occur  in  bursts.  This  dissertation  examines 
the  performance  of  some  of  the  more  complicated  coding  tech¬ 
niques  by  using  a  model  that  is  representative  of  a  real 
channel. 

This  model  describes  the  distribution  of  the  errors  in 
time.  It  relies  upon  the  concept  of  a  renewal  channel  (a 
channel  in  which  the  occurrence  of  an  error  removes  all 
memory  of  previous  errors).  The  model  introduced  in  this 
dissertation  is  shown  to  have  the  following  properties: 
ability  to  match  the  error  distributions  recorded  on  differ¬ 
ent  types  of  channels;  ease  of  selection  of  model  parameters; 
simplification  of  code  evaluation;  stationarity;  and  ergo- 
dicity . 

Several  error  control  techniques  were  evaluated.  These 
included  recurrent  codes,  concatenated  codes,  detect  and  re¬ 
transmit  codes  with  delayed  retransmissions  and  interleaved 
codei..  In  addition,  the  concept  of  space  diversity  coding 
is  introduced.  This  technique  takes  advantage  of  the  fact 
that  while  errors  may  occur  in  bursts  on  a  given  channel, 
bursts  of  errors  on  different  channels  may  occur  independ¬ 
ently.  This,  then,  encourapes  the  user  to  "code  across 
channels . 

While  the  evaluation  of  codes  is  the  primary  objective 
of  this  work ,  some  thought  is  given  to  the  problem  of  re¬ 
cording  the  error  statistics  to  which  the  model  is  matched. 

The  theory  of  extremes  is  shown  to  require  less  data,  when 
estimating  the  average  error  rate,  than  simplv  counting  ine 
number  of  errors  in  a  given  number  of  transmitted  bitr. . 

Two  small  simulations  were  conducted  to  confirm  this  con¬ 
clusion.  Theoretical  consideration  is  piven  to  the  esti¬ 
mators  of  the  parameters  in  the  extreme  value  distributions 
for  the  case  of  both  a  small  and  a  larpe  number  of  samples. 
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III-4.  "Nonparametric  Detection  Using  Extreme- Value  Theory" 
by  Laurence  B.  Milstein  -  1968 

Abstract 


This  pacer  concerns  itself  with  the  detection  of  a  binary 
signal  in  additive,  but  statistically  unknown,  noise.  The 
signal  will  be  either  a  constant  signal,  or  a  slowly  fading 
signal.  The  noise  will  be  arbitrary  except  for  the  one  res¬ 
triction  that  its  probability  density  function  exhibit  some 
type  of  exponential  behavior  on  its  "tails". 

The  detector  will  be  based  upon  Gumbel's  extreme-value 
theory  (EVT).  Extreme-value  theory  is  a  branch  of  mathemati¬ 
cal  statistics  which  considers  the  asymptotic  distributions 
of  the  maximum  and  minimum  samples  from  sets  of  independent 
and  identically  distributed  random  variables.  This  theory 
will  be  used  to  obtain  estimates  of  the  optimum  threshold 
and  the  probability  of  error  of  a  binary  detector.  Confi¬ 
dence  intervals  are  obtained  for  all  estimates. 

A  comparison  is  made  between  the  EV~  detector  and  another 
nonparametric  detector,  one  which  is  based  upon  the  rank  test. 
It  is  shown  that  in  certain  conditions,  the  EVT  detector  be¬ 
comes  identical  to  the  Neyman- Pearson  detector,  and  therefore 
will  outperform  the  rank  or  anv  other  nonparametric  detector. 

When  the  signal  fades,  it  is  shown  that  the  FVT  detector 
becomes  adaptive  and  can  track  the  fade.  Computer  simulations 
are  run  for  a  fading  signal,  and  the  results  verify  the  theory. 

Finally,  while  the  above  results  are  obtained  with  the 
help  of  an  initial  learning  period,  a  study  is  made,  for  the 
case  of  detecting  a  constant  signal  in  additive  noise,  of  the 
performance  of  the  detector  when  the  learning  period  is  re¬ 
moved.  It  is  shown  that  for  low  error  rates,  the  rst : 
will  converge  to  values  close  to  those  obtained  whe:.  the 
learning  period  is  present.  A  computer  simulation  is  run 
for  this  case,  and  again  the  results  verify  the  therrv. 
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1 1 1 -5 .  ''On  the  Transm-'ssion  of  Information  Over  the  Gaussian 
and  Related  Channels" 
by  Leonard  Schiff  -  1968 

Abstract 


This  thesis  is  concerned  with  the  transmission  of  infor¬ 
mation  from  a  digital  source  over  the  Gaussian  channel  and 
certain  closely  related  channels.  The  vector  space  point  of 
view  is  emphasized  throughout  the  work  so  that  all  transmit¬ 
ted  signals  are  considered  equivalent  to  vectors  in  a  finite 
dimensional  space. 

The  first  part  of  the  thesis  is  concerned  with  binary 
signaling  over  the  Gaussian  channel  such  that  every  T  sec¬ 
onds  the  transmitter  emits  the  sum  of  m  biphase  modulated 
signals.  This  signaling  scheme  is  considered  with  various 
types  of  receivers,  with  different  transmitter  power  con¬ 
straints  and  both  with  and  without  coding  (redundancy). 

The  transmission  system  performance,  under  the  various  con¬ 
ditions,  is  evaluated  by  bounding  the  error  probability 
both  from  above  and  below.  The  results  have  application 
in  the  study  of  binary  synchronous  multiplex  systems  and 
binary  transmissions  (using  error  correction  codes)  at 
data  rates  higher  than  the  Nvquist  rate. 

Transmission  of  information  over  the  Gaussian  channel 
at  very  low  signal  to  noise  ratios  (i.e.,  the  very  noisy 
continuous  channel)  is  also  considered.  Results  are  ob¬ 
tained,  both  with  and  without  receiver  quantization,  that 
are  analagous  to  results  previously  obtained  for  the  very 
noisv  amplitude-discrete  channel. 

The  final  section  of  this  thesis  is  concerned  with 
the  incoherent  channel  (and  an  extension  of  this  channel 
termed  the  "generalized  incoherent"  channel)  for  both 
high  and  low  signal  to  noise  ratios  and  both  with  and 
without  a  bandwidth  constraint.  For  the  band  unlimited 
charnel  results  are  obtained  that  are  similar  to  results 
alrt.^dy  known  for  the  use  of  orthogonal  signals  Transmit¬ 
ted  over  the  Gaussian  channel. 
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II 1-6 .  "Burst-Error  Correction" 

by  John  Dewey  Bri dwell  -  1968 

Abstract 

This  dissertation  is  concerned  with  encoding  of  bi.ary 
data  such  that  errors  introduced  by  a  burst -noise  channe' 
may  be  corrected.  Both  single-burst-correcting  (SBC)  rnd 
multiple-burst-correcting  (HBC)  codes  are  considered.  ?‘>e 
concept  of  burst  distance  is  defined  and  the  burst-correct¬ 
ing  properties  of  a  code  relative  to  its  burst  distance  are 
subsequently  developed. 

A  class  of  SBC  codes  related  to  Gilbert  codes  is  anal¬ 
yzed.  A  trade-off  is  shown  to  e;  ...st  between  the  block 
length  and  the  SBC  capability  for  a  code  from  this  cl&s*. 
The  exact  SBC  ability  of  Gilbert  codes  is  also  derived. 

Product  codes  are  shown  to  have  MBC  capabilities,  "he 
MBC  properties  of  product  codes  are  found  from  the  para¬ 
meters  of  the  subcodes.  A  class  of  product  codes  is  de¬ 
fined  such  that  single,  triple  and  quadruple  bursts  as 
well  as  five  single  errors  within  a  block  are  correctible. 

A  simple  decoding  algorith  given  for  this  class  of 
codes,  thus  indicating  that  product  codes  may  be  well- 
suited  to  practical  applications. 
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IIJ-7.  ''Error  Bounds  fr-t  the  Binary  Input  Gaussian  Noise 
Channel  with  Quantization" 
by  Edward  A.  Walvick  -  19G9 

Abstract 

The  problem  considered  in  this;  dissertation  is  the  deter 
mination  of  seme  bounds  on  the  probability  of  error  for  trars 
mission  ever  a  discrete  time,  additive  Gaussian  noise  channel 

The  input  to  the  channel  is  a  stream  of  signals,  each  of 
which  is  a  random  variable,  which  has  amplitude  +A  or  -A,  The 
output  is  a  stream  cf  random  variables  each  or  whi ch  has  a 
Gaussian  distribution  with  mean  +A  and  unit  variance.  The 
output  is  passed  through  an  L-level  quantizer.  Also  a  noise¬ 
less  feedback  link  may  or  mav  not  be  available  depending  on 
the  svstem  analyzed. 

Capacity  is  determined,  for  a  given  A,  as  a  function  of 
the  number  of  levels  of  quantization.  Also,  for  a  given 
number  of  levels-foptimum  division  of  output  levels  is  con¬ 
sidered.  Bounds  on  the  error  exponent  for  block  coding  are 
also  found  as  a  function  of  the  number  of  quantization  levels 
for  no  feedback  (upper  and  lower  bounds),  information  feed¬ 
back  (lower  bPund)  and  decision  feedback  (upper  and  lower 
bounds ) . 

The  results  indicate  that  considerable  deterioration 
in  performance  is  obtain’d  by  makinp  hard  decisions  on  the 
output  signals  (two  level  quantization).  Further,  for  a 
considerable  range  of  A,  st  least  half  the  possible  improve¬ 
ment  in  performance  obtainable  by  not  quantizing  the  output 
at  all,  is  obtained  by  three-level  quantization.  However, 
four  level  quantization  is  suggested  as  performance  is  then 
less  critically  dependent  on  the  location  of  the  thresholds 
which  divide  the  quantization  levels  than  with  three  level 
quantization. 
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II 1-8 .  "Rate-Distortion  Functions  for  Correlated  Gaussian  Sources" 
by  Barry  J,  Bunin  -  1970 

Abstract 

The  encoding  of  sn  analog  signal  into  binary  digits  is  a 
problem  of  ma/ior  importance  in  modem  communications.  The 
exact  specification  of  such  signals  reauires  in  general  an  in¬ 
finite  number  of  binary  digits.  Fortunately  an  exact  specifi¬ 
cation  is  rarely  needed.  Instead  we  are  usually  satisfied  if 
the  signal  we  reconstruct  from  the  binary  digits  is  close  to 
our  original  signal c  The  relevant  problem  becomes  the  speci¬ 
fication  of  an  analog  signal  to  within  some  tolerance  with  as 
few  binary  digits  as  possible. 

Shannon's  rate-distortion  function  tell  us,  for  a  given 
tolerance,  the  minimum  number  of  binary  digits  that  must  be 
used. 


In  this  dissertation  we  study  this  rate-distortion  func¬ 
tion  for  the  case  where  our  tolerance,  or  distortion,  is 
measured  by  the  familiar  mean  square  error,  and  where  the 
analog  signal  to  be  encoded  consists  of  a  sequence  of  corre¬ 
lated  Gaussian  random  variables. 

Shannon,  in  his  original  paper,  "The  Mathematical  Th-ory 
of  Communication,"  derived  the  rate-distortion  function  for 
an  independent  symbol  Gaussian  source.  Later,  Kolmogorov 
gave  the  rate-distortion  function  for  a  correlated  Gaussian 
source,  without  derivation.  In  Chapter  II,  we  present  this 
result  with  a  derivation. 

In  Chapter  III  we  consider  communication  systems  with  a 
noiseless  feedback  link.  It  is  shown  that  in  such  systems, 
the  minimum  mean  square  error  promised  by  rate-distortion 
theory  can  actually  be  achieved,  with  simple,  known  techniques. 

To  achieve  the  minimum  mean  square  error  with  feedback 
it  is  necessary  to  introduce,  in  the  limit,  infinite  delays. 

Ey  using  finite  delays,  we  fall  short  of  achieving  this  mini¬ 
mum.  In  Chapter  IV  the  rate  of  convergence  to  the  minimum 
mean  square  error  as  a  function  of  delay  was  investigated  for 
a  first  or^er  Markov  source.  The  convergence  was  shown  to 
behave  as  ^  where  N  is  the  block  length . 

Next,  in  Chapter  V,  an  upper  bound  on  the  convergence  to 
the  minimum  mean  square  error  is  found,  when  the  feedback  link 
is  removed.  When  a  noiseless  channel  is  used,  the  convergence 
is  upper  bounded  by  a  function  that  behaves  as  (log  N/N)1'  , 
and  when  a  noisy  channel  is  used,  by  (log  N/N)l/3. 
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In  Chapter  VI,  the  rate-distortion  function  is  derived 
for  the  source  whose  output  is  observed  only  after  it  is 
corrupted  by  correlated  noise.  The  derivation  is  performed 
with  block  length  as  a  parameter.  This  enables  us  to  use 
the  results  of  Chapter  IV  and  V,  to  upper  bound  the  conver¬ 
gence  to  the  minimum  mean  square  error,  as  encoding  delay 
increases.  When  the  noise  is  uncorrelated ,  the  convergences 
are  the  same  as  given  above. 

Finally,  in  Chapter  VII,  the  rate-distortion  functions 
of  the  class  of  nth  order  Markov  sources  are  studied.  It  is 
shown  that  these  sources  are  equivalent  to  uncorrelated 
Gaussian  sources,  for  bit  rates  greater  than  a  certain  bit 
rate,  denoted  R  .  .  The  rate  R  .  occurs  in  the  interval 
(o,n)  where  n  iS^he  order  of  tPfenMarkov  source.  Further, 
the  equivalent  uncorrelated  source  can  be  produced  by 
following  the  source  with  a  differential  predictive  system. 

A  differential  PCM  system,  and  an  adaptive  quantizing 
system  were  found  to  perform  within  4.34dh  of  the  rate- 
distortion  function  bounnd. 
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III —9 .  "A  Study  of  Lee  Metric  Codes" 
by  Chung- Yaw  Chiang  -  1970 

Abstract 


This  dissertation  is  mainly  concerned  with  channels  and 
codes  for  the  Lee  metric.  The  last  chapter  deals  with  codes 
for  correcting  restricted-magnitude  Hamming  (RMH)  errors. 

An  introductory  chapter  reviews  digital  communication 
systems,  block  codes  and  literature  related  to  this  disser¬ 
tation. 

Several  memoryless  channel  models  for  the  Lee  metric 
codes  are  given  in  Chapter  2,  This  chanter  is  also  con¬ 
cerned  with  the  general  properties  of  the  linear  Lee  metric 
codes.  Attention  is  focused  on  the  minimum  Lee  distance  of 
a  linear  code. 

Chapter  3  deals  with  negacyclic  Lee  metric  random-error- 
correcting  codes.  The  negacyclic  Lee  metric  codes  found  by 
E.  R.  Berlekamp  (1968)  are  modified.  It  results  in  a  class 
of  cyclic  Lee  metric  codes.  The  information  rate  of  the 
cyclic  and  negacyclic  Lee  metric  codes  >  discussed. 

Chapter  4  deals  with  cvclic  and  negacyclic  Lee  metric 
burst-error-correcting  codes.  A  class  of  single-burst- 
error-  correcting  codes,  based  on  Wyner's  low-density-burst- 
correcting  codes,  is  derived.  A  class  of  tensor  product 
codes  is  shown  to  be  multiple-burst -error-correcting  codes. 

Several  classes  of  codes  for  the  RMH  error  channel  are 
derived  in  Chapter  5.  This  type  of  channel  was  considered 
earlier  by  A.  D.  Wyner  (1966)  and  J.  K.  Wolf  (1969). 
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III-10.  "The  Time  Dispersive  Channel  as  a  Linear  Encoder" 
by  Theodore  J.  Klein  -  1970 

Abstracts 


The  problem  considered  in  this  dissertation  is  that  of 
reducing  intersymbol  interference  caused  by  time  dispersive 
channels.  This  is  not  a  new  problem,  and  much  recent  work 
has  centered  on  the  use  of  the  Tanned  Delay  Line  (TDL)  equal¬ 
izer.  The  conventional  apnroach  is  to  choose  the  no  disper¬ 
sion  channel  as  the  desired  channel,  and  then  to  minimize 
some  measure  of  the  intersymbol  interference. 

The  approach  described  in  this  dissertation  recognizes 
the  encoding  properties  of  time  disnersive  channels.  These 
channels  process  the  transmitted  data  in  much  the  same  wav 
as  the  generator  of  a  cyclic  algebraic  code.  Two  methods 
of  attack  are  taken.  In  the  first  method  the  code  generator 
coefficients  are  used  as  the  desired  response  for  an  other¬ 
wise  conventional  TDL  equalizer.  This  method  is  termed  the 
Coded  Fqualizer  method.  Transmission  of  k  q-arv  symbols 
through  the  channel  in  cascade  with  the  coded  equalizer  re¬ 
sults  In  a  code  word.  Thus,  error  correction  can  be  ob¬ 
tained  with  an  ordinary  algebraic  decoder,  and  without 
transmission  of  parity  symbols.  In  the  second  method  the 
channel  encoding  is  accepted  without  further  processing  by 
a  TDL  equalizer,  and  is  subsequently  decoded  by  a  channel 
decoding  matrix.  This  matrix  is  designed  to  minimize  the 
additive  noise  variance  subject  to  the  constraint  that  In¬ 
tersymbol  interference  be  eliminated.  A  modification  of 
this  method  allows  trading  computation  time  for  a  limited 
amount  of  intersymbol  interference. 

Upper  bounds  on  the  probability  of  error  are  derived 
for  both  methods.  These  bounds  are  in  the  form  of  easilv 
calculated  error  functions.  The  parameters  of  code  block 
length,  number  of  information  symbols  per  block,  and  alpha¬ 
bet  size  appear  explicitly. 

Computer  simulations  confirm  the  derived  bounds  and 
show  that  under  certain  conditions  orders  of  magnitude 
improvement  in  error  rate  can  be  obtained. 
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IV.  PAPERS 


IV-1.  A.  I.  Liff  and  J.  K.  Wolf  "On  the  Optimum  Sampling  Rate 
for  Discrete-Time  Modeling  of  Continuous-Time  Systems" 
IEEE  Transactions  on  Automatic  Control,  Vol.  AC-11, 
pp.  £66-296,  April  1966. 

Abstract 


The  sensitivity  effect  in  the  discrete-time  modeling  ot 
continuous -time  systems  in  digital  identification  schemes 
is  considered  as  a  function  of  the  sampling  interval  T.  It 
is  shown  that  the  common  assumption  that  the  higher  the  sam¬ 
pling  rate  the  better  a  discrete-time  model  represents  a 
continuous-time  system  is  not  true  in  general.  Rather,  it 
is  shown  that  an  optimum  sampling  rate  exists  which  minimizes 
the  effect  of  estimation  errors.  Experimental  results  are 
presented  which  confirm  the  existence  of  this  optimum  sam¬ 
pling  interval.  Close  confirmation  is  found  between  the 
theoretically  predicted  optimum  sampling  interval  and  the 
experimental  results. 
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IV-2.  K.  N.  Levitt  and  J.  K.  Wolf,  "Cyclically  Orthogonal  Sequences 
and  An  Application  to  Asynchronous  Multiplexing"  presented  at 
the  1966  IEEE  International  Symposium  on  Information  Theory, 
Los  Angeles,  California,  Feb.  1966. 

Abstract 


A  set  of  periodic,  binary  sequences  is  said  to  be  cyc¬ 
lically  orthogonal  if  every  pair  of  seouences  has  a  cross- 
correlation  function  which  is  identically  zero  for  all  time 
shifts.  Several  sets  of  such  sequences  are  presented  and 
are  applied  to  the  problem  of  designing  carriers  for  am 
asynchronous  multiplex  sy3tem.  The  probability  of  error 
for  the  system  is  calculated  and  compared  to  other  choices 
of  carriers. 
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IV-3.  J.  K.  Wolf,  "Signal  Design  for  Multiplex  Carriers"  presented 
at  the  1966  IEEE  Communications  Conference,  Philadelphia, 
Pennsylvania,  June  1966  (Conference  Proceedings,  p.  17R) 

Abstract 


Systematic  procedures  for  the  design  of  spread  spectrum 
and  pulse  address  signals  are  considered.  The  performance 
of  systems  using  various  types  of  carriers  is  derived.  The 
design  techniques  are  based  upon  algebraic  coding  theorv. 
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IV-4 .  K.  Levitt  and  J.  K.  Wolf,  "A  Class  of  Non-Linear  Error 

Correcting  Codes  Based  Upon  Interleaved  Two- Level  Sequences" 
IEEE  Transactions  on  Information  Theorv,  Vol.  IT-13, 
pp.  335-336,  April  1967. 

Abstract 


In  this  correspondence,  we  consider  nonlinear  binary 
error-correcting  codes  based  upon  the  n-fold  interleaving 
of  a  periodic  binary  sequence  of  length  L  with  out  of 
phase  (normalized)  autocorrelation  -  /L.  Examples  of 

binary  sequences  with  such  out  of  phase  autocorrelation 
(termed  two- level  seouences )  are:  1)  maximal-lenpth 
linear  shift-register  sequences,  2)  twin-prime  seouences, 
3)  quadratic  residue  sequences,  and  4)  Hall  sequences. 
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IV—  5 .  L.  Shaw  and  G.  Robinson,  "Invariant  Estimation  of  Stochastic 

System  Parameters"  presented  at  the  IFAC  Symoosium  on  "Problems 
of  Identification  in  Control  Systems,"  Prague,  Czechoslovakia, 
June  1967. 

Abstract 


This  paper  deals  with  the  estimation  of  the  coefficients 
of  a  finite-order  autorepression  equation.  A  class  of  esti¬ 
mates  are  investigated  based  upon  the  conditional  risk,  i.e., 
the  expected  loss  given  the  true  parameter  values.  Both  an 
estimator  and  a  cost  function  are  found  such  that  the  condi¬ 
tional  risk  is  independent  of  the  true  parameter  values.  It 
is  not  known  whether  the  resulting  estimates  yield  the  mini¬ 
mum  conditional  risk  attainable. 
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IV-6.  L.  Schiff,  "Synchronous  Binary  Multiplex  Systems" 
1967  International  Information  Theory  Conference, 
San  Remo,  Italy,  September  1967. 

Abstract 


A  synchronous  binary  multiplex  communication  system  is 
considered  whereby  every  T  seconds  the  transmitter  emits  the 
linear  sum  of  m  biphase  modulated  sub  carriers  s,(t).  The 
power  in  each  s.(t)  is  constrained  to  be  less  thin  some  fixed 
constant.  The  channel  is  assumed  to  have  a  bandwidth  W  =  ^ 

and  to  add  Gaussian  white  noise  to  the  transmitted  signal. 

The  performance  of  two  receiver  structures  are  analyzed 
in  detail.  The  first  receiver  structure  independently  demodu¬ 
lates  the  m  binary  messages  using  m  matched  filters.  If 
Y  =  -  <  1,  orthogonal  sub  carriers  result  in  an  error  proba¬ 
bility  which  goes  to  zero  exnonentiallv  with  n.  £or  y  >  1* 
if  the  s.(t)  are  chosen  as  a  mapping  of  the  m  =  2  code  words 
of  the  binary  group  code ,  the  error  probabi lity  remains  non¬ 
zero  even  when  n  *►  •.  However,  this  error  probability  is 
shown  to  be  smaller  than  the  average  error  probability  pro¬ 
duced  by  a  random  choice  of  the  s.(t). 

l 

A  second  receiver,  that  yields  a  minimum  probability  of 
error  on  the  joint  decision  of  the  m  binarv  digits  is  anal¬ 
yzed.  By  a  random  coding  bound  it  is  shown  that  for  any  y, 
a  set  of  s.(t)  exist  such  that  the  probability  of  error  ap¬ 
proaches  zero,  exponentially  with  n.  The  reliability  is 
calculated  and  is  shown  to  be  close  to  the  reliability  for 
the  orthogonal  case.  Tor  1  <  Y  <2,  a  semi -orthogonal  choice 
for  the  carriers  s.(t)  is  shown  to  have  a  reliability  equal 
to  that  for  the  orthogonal  case  for  low  signal-to-noise  ratio. 
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IV-7 .  J.  K.  Wolf,  "Decoding  of  Base-Choudhuri  Codes  and  Prony's 

Method  of  Cuive  Fitting,"  IEFE  Transactions  on  Information 
Theory ,  Vol.  IT-13,  No.  4,  p.  608,  October,  1967. 

Abstract 


The  literature  is  filled  with  exarrmles  of  a  common  set 
of  equations  which  arise  in  two  or  more  diverse  applications. 
The  purpose  of  this  correspondence  is  to  point  out  that  such 
a  situation  has  occurred  in  the  two  fields  of  1)  algebraic 
coding  theory,  and  2)  curve  fitting. 
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IV- 8.  B.  J.  Masnick  and  J.  K.  Wolf,  "On  Linear  Unequal  Error 

Protection  Codes",  IEEE  Transactions  on  Information  Theory, 
Vol.  IT-13,  No.  4,  October  1967. 

Abstract 


The  class  of  codes  discussed  in  this  paper  has  the  property 
that  its  error-correction  capability  is  described  in  terms  of 
correcting  errors  in  specific  digits  of  a  code  word  even  though 
other  digits  in  the  code  mav  be  decoded  incorrectly.  To  each 
digit  of  the  code  words  is  assigned  an  error  protection  level 
f..  Then  if  f  errors  occur  in  the  reception  of  a  code  word, 
axl  digits  which  have  protection  f .  >  f  will  be  decoded  correctly 
even  though  the  entire  code  word  mav  not  be  decoded  correctly. 

Methods  for  synthesizing  these  codes  are  described  and  il¬ 
lustrated  by  examples.  One  method  of  synthesis  involves  com¬ 
bining  the  parity  check  matrices  of  two  or  more  ordinary  random 
error  correcting  codes  to  form  the  parity  check  matrix  of  the 
new  code.  A  decoding  algorithm  based  upon  the  decoding  algor¬ 
ithms  of  the  component  codes  is  presented.  A  second  method  of 
code  generation  is  described  based  upon  the  observation  that 
for  a  linear  code,  the  columns  of  the  parity  check  matrix  cor¬ 
responding  to  the  check  positions  must  span  the  column  space 
of  the  matrix. 

Upper  and  lower  bounds  are  derived  for  the  number  of  check 
digits  required  for  such  codes  and  tables  of  numerical  values 
of  these  bounds  are  presented. 
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IV-9.  D.  Calabro  and  J.  K.  Wolf,  "On  the  Synthesis  of  Two- 

Dimensional  Arrays  with  Desirable  Correlation  Propert'es" 
Information  and  Control,  Vol,  11,  pp.  537-560,  November 
196  7.  Also  presented  at  1967  International  Symposium  on 
Information  Theory,  San  Remo,  Italy , Sept ember,  1967. 

Abstract 


Considerable  effort  has  been  devoted  in  the  literature 
to  the  synthesis  of  one-dimensional,  periodic,  binary  and 
nonbinary  sequences  having  small  values  for  their  out-of 
phase  autocorrelation  functions.  This  paper  considers  the 
synthesis  of  two-dimensional,  periodic,  binary  and  nonbinarv 
sequences  (arrays)  which  exhibit  similar  properties  for 
their  two-dimensional  autocorrelation  functions.  These 
arrays  may  have  future  application  in  the  areas  of  optical 
signal  processing,  pattern  recognition,  etc. 

Various  procedures  are  presented  for  the  synthesis  of 
such  arrays.  Two  perfect  binary  arravs  and  an  infinite 
class  of  perfect  nonbinarv  arravs  are  given.  A  class  of 
binary  arrays  is  presented  which  are  the  two-dimensional 
analogue  of  the  quadratic  residue  secuences  and  are  shown 
to  have  out-of-phase  autocorrelation  of  -1  or  to  alternate 
between  +1  and  -3.  The  perfect  maps  of  Gordon  are  shown 
to  have  all  values  of  out-of-phase  correlation  equal  to 
-1.  Other  methods  of  constructing  arravs  based  upon  good 
one-dimensional  sequences  are  also  discussed. 

Synthesis  procedures  are  given  for  constructing  pairs 
of  arrays  such  that  their  corss-correlation  is  identically 
zero  for  all  shifts  and  in  addition  individually  have  good 
autocorrelation  functions.  Examples  are  given  for  the 
various  synthesis  procedures. 
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IV-10.  M.  Muntner  and  J.  K.  Wolf,  "Predicted  Performance  of  Error- 
Control  Techniques  Over  Real  Channels",  IEEE  Transactions 
on  Information  Theory,  Vol.  IT-14,  pp.  640-650,  September 
1968. 

Abstract 


The  clustering  of  errors  on  real  channels  seriously 
complicates  the  task  of  evaluating  the  performance  of 
error-control  techniques.  A  model  is  introduced  that, 
in  addition  to  having  the  ability  to  match  erroi  distri¬ 
butions,  greatly  simplifies  the  task  of  code  evaluations. 
Having  selected  the  parameters  of  the  model  by  matching 
the  statistics  recorded  by  Townsend  and  Watts  on  the 
switched  telephone  network,  the  model  is  shown  to  pre¬ 
dict  the  results  of  tests  of  an  interleaved  burst-error- 
correcting  code.  The  utilitv  of  the  model  is  demonstra¬ 
ted  in  the  analysis  of  error-detection  codes  with  delayed 
retransmissions . 
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IV-11.  L.  Schiff,  'The  Asymptotic  Error  Probability  for  Transmission 

of  Orthogonal  Signals  Over  the  Generalized  Incoherent  Channel," 
IEEE  Transactions  on  Information  Theory,  Vol.  IT-15,  pp.  48-52, 
January  1969. 

Abstract 


The  probability  of  error  for  the  transmission  of  one  of 
M - orthogonal ,  equally  likelv,  equal-energy  signals  over  the 
white  Gaussian  noise  channel  is  known.  For  large  values  of 
T  it  is  shown  that  this  result  is  asymptotically  equivalent 
to  the  error  probability  for  the  same  signal  set  transmitted 
over  a  different  channel,  termed  here  the  generalized  inco¬ 
herent  channel  of  order  v.  For  v  =  2  this  channel  is  iden¬ 
tical  to  the  phase  incoherent  channel.  This  problem,  for 
larger  values  of  \),  also  arises  in  a  number  of  other  applica¬ 
tions,  some  of  which  are  given. 
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IV-12.  L.  Schiff  and  J.  K.  Wolf,  "Hiph  Speed  Binary  Data  Transmission 
Over  the  Additive  Band-Limited  Caussian  Channel,"  IFFF  Trans¬ 
actions  on  Information  Theory,  Vol.  IT-15,  pp.  287-2S5, 

March  1969. 

t  • 

Abstract 


The  transmission  of  the  linear  sum  of  m  hinhase  modulated 
signals  in  a  time  interval  T  is  considered  for  an  additive 
Oaussian  white  noise  channel  of  bandwidth  ft'.  Previous  analysis 
consider  the  case  where  m  <  n  =  2('.T,  In  this  paner,  the  pro¬ 

bability  of  error  is  derived  for  m  >  it,  a  situation  which 
arises  when  the  channel  bandwidth  is  insufficient  to  support 
the  data  rate. 

Two  distinct  problems  are  considered.  In  the  first, 
termed  uncoded  transmission,  all  m  sitmals  are  independently 
biphase  modulated.  It  is  shown,  for  this  case,  that  if  the 
channel  sipnal-to-noise  ratio  increases  linearlv  it,  the 
error  probability  can  be  made  to  pc  to  zero  approximately 
exponentially  in  it  for  anv  value  of  m/n. 

In  the  second  problem,  termed  coded  transmission,  only 
k  ^  m  of  the  sipnals  are  independentlv  modulated.  (The  re- 
maininp  [m  -  f?)  sipnals  carry  redundant  information).  By 
usinp  a  suboptimum  receiver,  it  is  shown  that  for  a  fixed 
channel  sipnal-to-noise  ratio  the  error  probability  poes  to 
0  exponentially  in  it  if  fc/rt  is  less  than  some  number  C*. 

For  high  sipnal-to-noise  ratio,  f *  is  preater  than  1,  a 
situation  which  could  not  occur  if  m  S  it. 
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IV-13.  Laurence  B.  Milstein,  Donald  L.  Schilling  and  Jack  K.  Wolf, 
"Robust  Detection  Using  Extreme-Value  Theory,"  IFFF  Trans- 
actions  on  Information  Theory,  Vol.  IT-15,  pp.  370-395, 

May,  1969. 

Abstract 


The  use  cf  extreme-value  theory  (FVT)  in  the  detection 
of  the  binary  signal  in  additive,  but  statistically  unknown, 
noise  is  considered.  It  is  shown  that  the  optimum  threshold 
and  the  probability  of  error  of  the  system  can  be  accurately 
estimated  by  using  EVT  to  obtain  properties  of  the  initial 
probability  density  functions  on  their  "tails".  Both  con¬ 
stant  signals  and  slowly  fading  signals  are  considered.  In 
the  case  of  a  fading  signal,  the  detector  becomes  adaptive. 
Detection  of  the  constant  signal,  both  with  and  without  an 
initial  learning  period,  is  studied  bv  comnuter  simulation. 


39 


IV-14.  j.  K,  Wolf,  "Adding  Two  Information  Symbols  to  Certain 

Nonbinary  BCH  Codes  and  Some  Applications,"  presented  at 
Symposium  on  Information  Theory,  Dubna,  USSR,  June  19-25, 
1969. 

Abstract 


This  paner  is  a  compendium  of  results  based  on  a 
simple  observation:  two  information  symbols  caN  be  appended 
to  certain  nonbinary  BCH  codes  without  affecting  the  guar¬ 
anteed  minimum  distance  of  these  codes.  We  give  two  for¬ 
mulations  which  achieve  this  result;  the  second  yields  in 
information  regarding  the  weights  of  coset  leaders  for  the 
original  BCH  codes. 

Single-error-correcting  Reed-Solomon  codes  with  the 
added  information  symbols  yield  perfect  codes  for  the 
Hamming  metric.  We  use  these  lengthened  Reed-Solomon 
codes  as  building  blocks  for  perfect  single-error-correct- 
in?  codes  in  another  metric. 
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IV-15 .  J,  K.  Wolf,  M.  I..  Shooman  and  R.  R.  Boorstyn,  ’’Algebraic 
Coding  and  Digital  Redundancy"  IEEE  Transactions  on 
Reliability,  Vol.  R-18,  pp.  91-10?,  August,  1969. 

Abstract 


The  techniques  of  coding  theory  are  used  to  improve  the 
reliability  of  digital  devices.  Redundancy  is  added  to  the 
device  by  the  addition  of  extra  digits  which  are  independently 
computed  from  the  input  digits.  A  decoding  device  examines 
the  original  outputs  along  with  the  redundant  outputs.  The 
decoder  may  correct  any  errors  it  detects,  not  correct  but 
locate  the  defective  logic  gate  or  subsystem,  or  only  issue 
a  general  error  warning.  Maiority  voting  and  parity  bit 
checking  are  introduced  and  the  computations  are  made  for 
several  binary  addition  circuits.  A  detailed  summary  of 
coding  theory  is  presented.  This  includes  a  discussion  of 
.lgebraic  codes,  binary  group  codes,  nonbinary  linear  codes, 
and  error  locating  codes. 
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IV-16.  B.  J.  Bunin  "Rate-Distortion  Functions  for  Gaussian  Markov 
Processes"  Bell  System  Technical  Journal,  vol.  48, 
pp,  3059-3074,  November  1969 . 

Abstract 


The  rate-distortion  function  with  a  mean  square  error 
distortion  criterion  is  investigated  for  a  class  of  Gaussian 
Markov  sources.  It  is  found  that  for  rates  greater  than  a 
certain  minimum,  the  rate-distortion  function  is  equivalent 
to  that  of  an  independent  letter  source.  This  minimum  rate 
was  found  to  be  less  than  n  bits  per  symbol,  where  n  is  the 
order  of  the  Markov  sequence.  Comoarisons  between  the 
rate-distortion  function,  and  two  quantizing  systems  are 
made. 
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IV-17 .  J.  D.  Bridwell  and  J.  K.  Wolf,  "Burst  Distance  and  Multiple- 
Burst  Correction"  Bell  System  Technical  Journal,  VoL.  49, 
po.  889-909,  May- June  1970. 

Abstract 


This  paper  is  concerned  with  burst  error,  burst  erasure 
and  combined  burst-error  and  burst-erasure  correction.  Part 
I  introduces  the  concept  of  burst  distance  and  subsequently 
develops  burst-correct inf?  properties  of  a  code  relative  to 
its  burs1-  distance.  Part  II  discussed  product  codes  for 
multipit  u*st  correction  (MBC).  The  MBC  properties  of  a 
product  or  two  codes  are  derived  from  the  properties  of  the 
original  codes.  The  correction  of  spot  errors  is  generalized 
to  multiple-spot  correction.  Theorems  are  presented  which 
strengthen  the  single-burst  correcting  (SBC)  properties  of 
some  codes.  A  class  of  codes  which  corrects  single,  triple 
and  quadruple  bursts  and  5  single  errors  is  developed,  and 
a  decoding  procedure  is  given.  Finally,  a  code  from  the  new 
class  of  MBC  codes  is  compared  with  three  other  MBC  codes. 
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IV-18. 


B.  J.  Bunin  and  J.  K.  Wolf,  "Convergence  to  the  Rate-Distortion 
Function  for  Gaussian  Sources"  Accented  for  publication  in  the 
IEEE  Transactions  on  Information  Theory,  1971. 

Abstract 

In  this  pater  we  derive  an  expression  for  the  minimum 
mean  squar’  error  achievable  in  encoding  t  samples  of  a 
stationary  correlated  Gaussian  ource.  It  is  assumed  that 
the  source  output  is  not  known  exactly,  but  is  corrupted 
by  correlated  Gaussian  noise.  The  expression  is  obtained 
in  terms  of  the  covariance  matrices  of  the  source  and  noise 
sequences.  It  is  shown  that  t  •,  the  result  agrees  with 
a  known  asymptotic  result,  which  is  expressed  in  terms  of 
the  power  spectra  of  the  source  and  noise. 
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This  report  snaaaarices  the  results  of  five  years  of  research  In  the  general 
areas  of  slgsbrolc  sy sterna  and  stochastic  ay a terns.  Topics  considered  related  to 
algobrale  system  include:  unequal  error  protection  eodos,  burst  and  multiple 
hurst  correcting  codes,  eodos  designed  for  the  Lso  metric,  improved  BCH  codes, 
decoding  of  BCH  codes,  systom  reliability  through  elgebralc  coding,  channel  modeling 
of  real  chaimele,  the  uao  of  a  time  dispersive  channel  aa  an  encoder  and  the  study 
of  certain  algebraic  sequences.  In  tlte  study  of  stochastic  systems  the  following 
topics  were  considered:  optimum  sampling,  invariant  estimators,  feedback  communica¬ 
tions,  adaptive  detection  and  loaning,  high  speed  binary  data  transmission  over 
a-bsnd-limited  channel,  multiple  access  communications  and  rato  distortion  functions 
for  stochastic  sources. 

9*ly  *  brief  svwnary  of  the  research  remits  is  presented.  However,  abstracts 
of  papers  and  dissertations  resulting  from  this  research  are  included. ' 
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